BUILD-IT: Intuitive plant layout mediated by natural interaction

BUILD-IT is a planning tool based on intuitive computer vision technology, supporting complex planning and configuration tasks. Based on real, tangible bricks as an interaction medium, it represents a new approach to Human Computer Interaction (HCI). It allows a group of people, seated around a table, to move virtual objects using a real brick as a interaction handler. Object manipulation and image display take place within the very same interaction space. Together with the image displayed on the table, a perspective view of the situation is projected on a vertical screen. The system offers all kinds of users access to state-of-the-art computing and visualisation, requiring little computer literacy. It offers a new way of interaction, facilitating team-based evaluation of alternative layouts. M. Fjeld, M. Bichsel & M. Rauterberg (1999): BUILD-IT: Intuitive plant layout mediated by natural interaction. Arbete Människa Miljö & Nordisk Ergonomi (Work, Human being, Environment), 1/99, pp. 49-56. Introduction Supporting natural behaviour in Human-Computer-Interaction (HCI) is getting increasingly important. We suggest a new concept to enhance human expression and to support cognitive processes by making them visible. To allow for a natural or direct way of task solving behaviour, we define a set of six design principles. These principles are then used as support to design an interaction tool called BUILD-IT. Based on tangible bricks as interaction handlers, this system enables users to interact with complex data in a direct way. We call our design concept the Natural User Interface (NUI). Outline of design principles As pointed out in the introduction, there is a need for a concept bringing together cognitive (here: goal related) and motor activity. Based on task analysis, action regulation theory (Hacker, 1994) is one possible concept to answer this need. We choose action regulation theory as the psychological basis for this work. Within this tradition, high importance is given to the concept of complete task. A complete task starts with a goal setting part, followed by three subsequent steps (Figure 1). task description goal setting control with feedback (observable) action planning Figure 1 A complete activity cycle in action regulation theory. In more detail, these four steps are: individual setting of goals, given by the task description, and later on, given by controlled feedback, taking on planning functions, selecting tools and preparing actions necessary for goal attainment, physical (or even mental) performance functions with feedback on performance pertaining to possible corrections of actions, and M. Fjeld, M. Bichsel & M. Rauterberg (1999): BUILD-IT: Intuitive plant layout mediated by natural interaction. Arbete Människa Miljö & Nordisk Ergonomi (Work, Human being, Environment), 1/99, pp. 49-56. controlled feedback on results and the possibility of checking the action results against goals. When computer users pursue an activity, their goal may be more or less clear. Their actions may be classified according to goal-relatedness. Kirsh and Maglio (1994) considered motor activity as being either epistemic or pragmatic. Pragmatic actions have the primary function of bringing the user physically closer to a goal. In contrast, epistemic actions are chosen to unveil hidden information or to gain insight that otherwise would require much mental computation. Hence, physical actions facilitate mental activity, making it faster and more reliable. Cognitive complexity may also be reduced by epistemic actions. Epistemic and pragmatic actions are, generally speaking, both present in task-solving behaviour. This applies to all levels of expertise. Independent of the level of expertise, pragmatic and epistemic actions are both necessary for successful task solving performance and should therefore be encouraged in the design of HCI tools. Pragmatic actions seem to come close to Hacker’s (1994) goal-driven actions. However, if no goal can be derived directly from a task description, the first part of solving the task is epistemic. In that case, a complete activity cycle starts with observable action, followed by goal setting and planning (Figure 2). goal setting control with feedback planning task description (observable) action Figure 2 A complete activity cycle in the case of epistemic action. In the rest of this paper, we make the abstraction that pragmatic, as well as epistemic action both can be represented by Figure 1. This means that the top and bottom of the cycle in Figure 1 should no longer be taken literally. That figure is meant to transport both the idea of complete pragmatic as well as the idea of complete epistemic actions. ___________________ 1 Knowledge-based 2 Practice-based M. Fjeld, M. Bichsel & M. Rauterberg (1999): BUILD-IT: Intuitive plant layout mediated by natural interaction. Arbete Människa Miljö & Nordisk Ergonomi (Work, Human being, Environment), 1/99, pp. 49-56. Now, the first three design principles for graspable interfaces can be outlined: Assure that mistakes only imply low risk so that epistemic behaviour is being stimulated, allow users to choose between epistemic (exploratory) and pragmatic (goal-oriented) actions, and support a complete regulation of pragmatic as well as epistemic behaviour. Coinciding action and perception spaces When manipulating objects in the real world, action space (hands and fingers) and perception space (the position of the object in the real world) coincide in time and space (Rauterberg, 1995). Hacker and Clauss (1976) proved that offering task-relevant information in the same space as where action takes place leads to increased performance. With a screen-keyboard-mouse user interface, there is a separation between these two spaces, given by the separation of inand output devices. An alternative approach to interface design (Rauterberg, 1995), is to let perception and action space coincide (Figure 3). task description goal setting control with feedback (observable) action planning Perception &